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Abstract 

In this paper, we propose three new notions of (even-order) tensor ranks, to be called the 
M-rank, the symmetric M-rank, and the strongly symmetric M-rank. We discuss the bounds 
between these new tensor ranks and the CP(CANDECOMP/PARAFAC)-rank and the sym¬ 
metric CP-rank of an even-order tensor. In particular, we show: (1) these newly defined ranks 
actually coincide with each other if the even-order tensor in question is super-symmetric; (2) 
the CP-rank and symmetric CP-rank for a fourth-order tensor can be both lower and upper 
bounded (up to a constant factor) by the corresponding M-rank, and the bounds are tight for 
asymmetric tensors. In addition, we manage to identify a class of tensors whose CP-rank can be 
easily determined, and the CP-rank of the tensor equals the M-rank but is strictly larger than 
the Tucker rank. Since the M-rank is much easier to compute than the CP-rank, we propose 
to replace the CP-rank by the M-rank in the low-CP-rank tensor recovery model. Numerical 
results suggest that when the CP-rank is not very small our method outperforms the low-n-rank 
approach which is a currently popular model in low-rank tensor recovery. This shows that the 
M-rank is indeed an effective and easily computable approximation of the CP-rank. 
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1 Introduction 


Tensor data have appeared frequently in applications such as computer vision [42] , psychometrics 
[15, 8], diffusion magnetic resonance imaging [13, 3, 36], quantum entanglement problem [17], spec¬ 
tral hypergraph theory [18] and higher-order Markov chains [29]. Tensor-based multi-dimensional 
data analysis has shown that tensor models can take full advantage of the multi-dimensionality 
structures of the data, and generate more useful information. A common observation for huge-scale 
data analysis is that the data exhibits a low-dimensionality property, or its most representative 
part lies in a low-dimensional subspace. To take advantage of this low-dimensionality feature of 
the tensor data, it becomes imperative that the rank of a tensor, which is unfortunately a notori¬ 
ously thorny issue, is well understood and computationally manageable. The most commonly used 
definition of tensor rank is the so-called CP-rank, where “C” stands for CANDECOMP while “P” 
corresponds to PARAFAC and these are two alternate names for the same tensor decomposition. 
The CP-rank to be introduced below, is the most natural notion of tensor rank, which is also very 
difficult to compute numerically. 

Definition 1.1 Given a d-th order tensor T G (j-;nixn 2 x-.xnd complex domain, its CP-rank 
(denoted as xai\kcp{J-)) is the smallest integer r exhibiting the following decomposition 

r 

T = ^ a^’* (g) (g) • • • (g) (1) 

i=\ 

where G C”* for k = l,...,d and i = l,...,r. Similarly, for a real-valued tensor T G 
M"'iX”- 2 X"-xnd^ its CP-rank in the real domain (denoted as rank®p(J ^)) is the smallest integer r such 
that there exists a real-valued decomposition (1) with G for k = 1,... ,d and i = 1,...,r. 

An extreme case is r = 1, where P is called a rank-1 tensor in this case. For a given tensor, finding 
its best rank-1 approximation, also known as finding the largest eigenvalue of a given tensor, has 
been studied in [30, 35, 10, 23, 21]. It should be noted that the CP-rank of a real-valued tensor 
can be different over M and C; i.e., it may hold that rankc>p(J^) < rank®p(J^) for a real-valued 
tensor P. For instance, a real-valued 2x2x2 tensor P is given in [26] and it can be shown that 
rank®p(J^) = 3 while rankcp(J^) = 2. In this paper, we shall focus on the notion of CP-rank in 
the complex domain and discuss two low-CP-rank tensor recovery problems; tensor completion and 
robust tensor recovery. 

Low-CP-rank tensor recovery problem seeks to recover a low-CP-rank tensor based on limited 
observations. This problem can be formulated as 

min rankc>p(A’), s.t. L{P) = b, (2) 
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where L : (C^ix"’ 2 ---x«'d —is a linear mapping and b € denotes the observation to X under L. 
Low-CP-rank tensor completion is a special case of (2) where the linear mapping in the constraint 
picks certain entries of the tensor. The low-CP-rank tensor completion can be formulated as 

min rankcpCT), s.t. Pq{X) = PniXo), (3) 

;reC"ix’»2'"X"d 


where Xq G i[^nixn 2 —xnd given tensor, O is an index set and 


[Pn{X)]i^^i2,...,id 


0, otherwise. 


In practice, the underlying low-CP-rank tensor data P G £^nixn 2 —xnd be heavily corrupted 
by a sparse noise tensor y. To identify and remove the noise, one can solve the following robust 
tensor recovery problem: 

min rankcp(T) + A||Z||o, s.i.y + Z = P, (4) 

y,.2eC"iX"2'"X"d 

where A > 0 is a weighting parameter and ||2^||o denotes the number of nonzero entries of Z. 

Solving (2) and (4), however, are nearly impossible in practice. In fact, determining the CP-rank 
of a given tensor is known to be NP-hard in general [16]. Worse than solving a common NP-hard 
problem, computing the CP-rank for small size instances remains a difficult task. For example, 
a particular 9x9x9 tensor is cited in [27] and its CP-rank is only known to be in between 18 
and 23 to this date. The above low-CP-rank tensor recovery problems (2) and (4) thus appear 
to be quite hopeless. One way out of this dismay is to approximate the CP-rank by some more 
reasonable objects. Since computing the rank of a matrix is easy, a classical way for tackling tensor 
optimization problem is to unfold the tensor into certain matrix and then resort to some well 
established solution methods for matrix-rank optimization. A typical matrix unfolding technique 
is the so-called mode-n matricization [22]. Specifically, for a given tensor P G 
denote its mode-n matricization by F (n), which is obtained by arranging the n-th index of P as the 
column index of F{n) and merging other indices of P as the row index of F{n). The Tucker rank 
(or the mode-n-rank) of P is defined as the vector (rank(T(l)),..., rank(T(d))). For simplicity the 
averaged Tucker rank is often used and denoted as 

1 

rankn(T') := - ^ rank(F(j)). 
i=i 

As such, rank„(T') is much easier to compute than the CP-rank, since it is just the average of 
d matrix ranks. Therefore, the following low-n-rank minimization model has been proposed for 
tensor recovery [31, 12]: 

min rankn(A’), s.t. L{X) = b, (5) 
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where L and b are the same as the ones in (2). Since minimizing the rank function is still difficult, 
it was suggested in [31] and [12] to convexity the matrix rank function by the nuclear norm, which 
has become a common practice due to the seminal works on matrix rank minimization (see, e.g., 
[11, 37, 6, 7]). That is, the following convex optimization problem is solved instead of (5): 


1 


d 


s.t.L{X) = b, 


( 6 ) 


where the nuclear norm ||X(j)||* is defined as the sum of singular values of matrix X{j). Some effi¬ 
cient algorithms such as alternating direction method of multipliers and Douglas-Rachford operator 
splitting methods were proposed in [31] and [12] to solve (6). 


However, to the best of our knowledge, the relationship between the CP-rank and the Tucker 
rank of a tensor is still unclear so far, although it is easy to see (from similar argument as in 
Theorem 3.3) that the averaged Tucker rank is a lower bound for the CP-rank. In fact, there is a 
substantial gap between the averaged Tucker rank and the CP-rank. In Proposition 3.4, we present 
annxnxnxn tensor whose CP-rank is n times the averaged Tucker rank. Moreover, in the 
numerical experiments we found two types of tensors whose CP-rank is strictly larger than the 
averaged Tucker rank; see Table 1 and Table 2. The theoretical guarantee of model (6) has been 
established in [41], which states that if the number of observations is at the order of 0(rn'^“^), then 
with high probability the original tensor with Tucker rank (r, r, • • • , r) can be successfully recovered 
by solving (6). Therefore, model (6) and its variants have become popular in the area of tensor 
completion. However, our numerical results show that unless the CP-rank is extremely small, (6) 
usually fails to recover the tensor; see Table 3. As a different tensor unfolding technique, the square 
unfolding was proposed by Jiang et al. [21] for the tensor rank-one approximation problem. This 
technique was also considered by Mu et al. [34] for tensor completion problem. Mu et al. [34] 
showed that when the square unfolding is applied, the number of observations required to recover 
the tensor is of the order of which is significantly less than the number needed by (6). 

Convexifying the robust tensor recovery problem (4) was also studied by Tomioka et al. [40] and 
Goldfarb and Qin [14]. Specifically, they used the averaged Tucker rank to replace the CP-rank of 
y and II2^11 1 to replace ||2^||o in the objective of (4). However, we observe that this model cannot 
guarantee that the resulting solution X is of low CP-rank as shown in Table 6. Other works on 
this topic include [38, 24, 25]. Specifically, [38] compared the performance of the convex relaxation 
of the low-n-rank minimization model and the low-rank matrix completion model on applications 
in spectral image reconstruction. [24] proposed a Riemannian manifold optimization algorithm for 
finding a local optimum of the Tucker rank constrained optimization problem. [25] studied some 
adaptive sampling algorithms for low-rank tensor completion. 


Note that in the definition of CP-rank, T G -g decomposed into the sum of asymmetric 

rank-one tensors. If T" is a super-symmetric tensor, i.e., its component is invariant under any 
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permutation of the indices, then a natural extension is to decompose J- into the sum of symmetric 
rank-one tensors, and this leads to the definition of symmetric CP-rank (see, e.g., [9]). 


Definition 1.2 Given a d-th order n-dimensional super-symmetric complex-valued tensor T, its 
symmetric CP-rank (denoted by lavikscpi^)) is the smallest integer r such that 

r 

T = a* (g) • • • (g) a*, 

d 


with a® G C”, i = 1,..., r. 


It is obvious that rankcp(J^) < rank 5 C'p(J^) for any given super-symmetric tensor T. In the 
matrix case, i.e., when d = 2, it is known that the rank and symmetric rank are identical. However, 
in the higher order case, whether the CP-rank equals the symmetric CP-rank is still unknown, 
and this has become an interesting and challenging open problem (see Comon et al. [9]). There 
has been some recent progress towards answering this question. In particular, Zhang et al. [44] 
recently proved that rank( 7 p(J^) = rank 5 ( 7 p(J^) holds for any d-th order super-symmetric tensor J- 
if rankcp(J^) < d. 

At this point, it is important to remark that the CP-rank stems from the idea of decomposing 
a general tensor into a sum of simpler - viz. rank-one in this context - tensors. The nature of 
the “simpler components” in the sum, however, can be made flexible and inclusive. In fact, in 
many cases it does not have to be a rank-one tensor as in the CP-decomposition. In particular, 
for a 2d-th order tensor, the “simple tensor” being decomposed into could be the outer product 
of two tensors with lower degree, which is d in this paper, and we call this new decomposition 
the M-decomposition. It is easy to see (will be discussed in more details later) that after square 
unfolding each term in the M-decomposition is actually a rank-one matrix. Consequently, the 
minimum number of such simple terms can be regarded as a rank, or indeed the M-rank in our 
context, to be differentiated from other existing notions of tensor ranks. By imposing further 
symmetry on the “simple tensor” that composes the M-decomposition, the notion of symmetric M- 
decomposition (symmetric M-rank), and strongly symmetric M-decomposition (strongly symmetric 
M-rank) naturally follow. We will introduce the formal definitions later. The merits of the M-rank 
are twofold. First, for some structured tensors, we can show, through either theoretical analysis or 
numerical verifications, that the M-rank is much better than the averaged Tucker rank in terms of 
approximating the CP-rank. Second, for low-CP-rank tensor recovery problems, the low-M-rank 
approach can improve the recoverability and our numerical tests suggest that the M-rank remain 
a good approximation to the CP-rank even in the presence of some gross errors. 

The main contributions of this paper are as follows. First, we introduce several new notions of 
tensor decomposition for the even order tensors, followed by the new notions of tensor M-rank, 
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symmetric M-rank and strongly symmetric M-rank. Second, we prove the equivalence of these 
three rank definitions for even-order super-symmetric tensors. Third, we establish the connection 
between these new ranks and the CP-rank and symmetric CP-rank. Specifically, we show that 
for a fourth-order tensor, both the CP-rank and the symmetric CP-rank can be lower and upper 
bounded (up to a constant factor) by the M-rank, and the bound is tight for asymmetric tensors. As 
a byproduct, we present a class of tensors whose CP-rank can be exactly computed easily. Finally, 
in the numerical experiments we show that the M-rank is equal to the CP-rank in all randomly 
generated instances and the low-M-rank method outperforms the low-n-rank approach for both 
tensor completion and robust tensor recovery problems. 

Notation. We use to denote the n-dimensional complex-valued vector space. We adopt calli¬ 
graphic letter to denote a tensor, i.e. A = (Mip 2 ---id)nixn 2 x...xnd- (denotes the space 

of d-th order ni x n 2 x • • • x dimensional complex-valued tensor. 7r(ii,i2,-- - ,id) denotes a 
permutation of indices Ad)- We use A-^ to denote the tensor obtained by permuting 

the indices of A according to permutation vr. Formally speaking, a tensor T" G ^nixn 2 x — xni -g 
called super-symmetric, if ni = n 2 = ... = and T = for any permutation vr. The space 
where n x n x ■ ■ ■ x n super-symmetric tensors reside is denoted by . We use (8> to denote 
d 

the outer product of two tensors; that is, for Mi G £nixn 2 x---xnd ^ ]^nd+ixnd+ 2 x---xnd+(.^ 

Ml (8) M 2 G C"lXn2X...xnd+, 

(Ml (8> M2)qi2---'jd+^ ~ 

2 M-rank, symmetric M-rank and strongly symmetric M-rank 

In this section, we shall introduce the M-decomposition (correspondingly M-rank), the symmetric 
M-decomposition (correspondingly symmetric M-rank), and the strongly symmetric M-decomposition 
(correspondingly strongly symmetric M-rank) for tensors, which will be used to provide lower and 
upper bounds for the CP-rank and the symmetric CP-rank. 

2.1 The M-rank of even-order tensor 

The M-decomposition of an even-order tensor is defined as follows. 

Definition 2.1 For an even-order tensor F G the M-deeomposition is to find some 

tensors M* G ^ ipyid+ix-xn 2 d i = 1,... ,r such that 

r 

jr = J2A^^B\ (7) 

i=l 
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The motivation for studying this decomposition stems from the following novel matricization tech¬ 
nique called square unfolding that has been considered in [21, 34, 43]. 

Definition 2.2 The square unfolding of an even-order tensor G ^nixn 2 —xn 2 d (denoted by M{J^) G 
£^(ni-nd)xind+i-n 2 d)J ig q matrix that is defined as 

M{T)ki ■= ^ii-i2d^ 

where 

d j-i 

k = '^^{ij — 1) ng -|- ii, 1 < ^ nj, 1 < j < d, 

j=2 q=l 

2d j-1 

^ = E n Uq -|- id+1, i<ij<'nj,d-\-l<j< 2d. 

j=d+2 q=d-\-l 


In Definition 2.2, the square unfolding merges d indices of J- as the row index of M[T), and merges 
the other d indices of J- as the column index of In this pattern of unfolding, we can see 

that the M-decomposition (7) can be rewritten as 

r 

i=l 

where a* = V{A^), 6® = V{B^) for i = I,..., r, and V{-) is the vectorization operator. Specifically, 
for a given tensor T G ( 7 mxn 2 ---xnd^ V{iF)k ■= J^h-id with 

d j-i 

k = '^^{ij — 1) Uq -|- ii, 1 < ij < Uj, 1 < j < d. 

3=2 q=l 

Therefore, the M-decomposition of T is exactly the rank-one decomposition of the matrix M{iF). 
Apparently, unless J- is super-symmetric, there are different ways to unfold the tensor T" by per¬ 
muting the 2d indices. Taking this into account, we now define two types of M-rank (namely, 
M“*'-rank and M“-rank) of an even-order tensor as follows. 

Definition 2.3 Given an even-order tensor G its M~ -rank (denoted &?/rank^-(T") J 

is the smallest rank of all possible square unfolding matrices, i.e., 

rankj^//-(T") = min rank (iW(T'jr)), (8) 

7ren(l,...,2d) 

where 11(1,..., 2d) denotes the set of all possible permutations of indices (1,..., 2d), and is 
the tensor obtained by permuting the indices of T according to permutation vr. In other words, 
[T) is the smallest integer r such that 

r 

i=l 
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holds for some permutation n G 11(1,..., 2d), G ^ qi ^ ^nh (ii, • • • , i 2 d) 

'7r(l,-- - ,2d), i = 1,... ,r. Similarly, the M'^-rank (denoted by ia.'nkj^+{F)) is defined as the largest 
rank of all possible square unfolding matrices: 

isrokj^+iF) = max rank . (10) 

7r£n(l,...,2£/) 

2.2 Symmetric M-rank and strongly symmetric M-rank of even-order super- 
symmetric tensor 

Note that if F is an even-order super-symmetric tensor, racakj^+{F) = rank^^^-(J^). In this case, we 
can simplify the notation without causing any confusion by using rankjvf(J^) to denote the M-rank 
of F. 

As we mentioned earlier, the decomposition (7) is essentially based on the matrix rank-one decom¬ 
position of the matricized tensor. In the matrix case, it is clear that there are different ways to 
decompose a symmetric matrix; for instance, 

ab~^ + baJ = -(a -|- 6)(a -|- h)~^ — -(a — h){a — b)~^. 

In other words, a given symmetric matrix may be decomposed as a sum of symmetric rank-one 
terms, as well as a sum of non-symmetric rank-one terms, however yielding the same rank: the 
minimum number of respective decomposed terms. A natural question arises when dealing with 
tensors: Does the same property hold for the super-symmetric tensors? The M-decomposition of a 
tensor is in fact subtler: the decomposed terms can be restricted to symmetric products, and they 
can also be further restricted to be super-symmetric. 

Therefore, we can define the symmetric M-rank and the strongly symmetric M-rank for even-order 
super-symmetric tensor as follows. 

Definition 2.4 For an even-order super-symmetric tensor F G its symmetric M-decomposition 
is defined as 

r 

F = '^B^®B\ 13* G = l,...,r. (11) 

i=l 

The symmetric M-rank of F (denoted by iax\ksM{^)) is the rank of the symmetric matrix M[F), 
i.e., la.'nksMi^) = rank(iW(T')) = rankjvf {F); or equivalently rsvaksM^^) is the smallest integer r 
such that (11) holds. 

In a similar vein, the strongly symmetric M-decomposition is defined as 

r 

= A* GS**",i = l,...,r, 

i=l 
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( 12 ) 


and the strongly symmetric M-rank of F (denoted by rankssMiF)) is defined as the smallest integer 
r such that (12) holds. 

The fact that the M-rank and the symmetric M-rank of an even-order super-symmetric tensor 
are always equal follows from the similar property of the symmetric matrices. (Note however the 
M-decompositions may be different). Interestingly, we can show that rank5'M(-7^) = i'ank 55 M(-^) 
for any even-order super-symmetric tensor F, which appears to be a new property of the super- 
symmetric even-order tensors. 


2.3 Equivalence of symmetric M-rank and strongly symmetric M-rank 

To show the equivalence of the symmetric M-rank and the strongly symmetric M-rank, we need to 
introduce the concept of partial symmetric tensors and some lemmas first. 

Definition 2.5 We say a tensor F G C"** partial symmetric with respect to indices {l,...,m}, 
m < d, if 

We use TTij G 11(1, ■ ■ ■ ,d) to denote the specific permutation that exchanges the i-th and the j-th 
indices and keeps other indices unchanged. 


The following result holds directly from Definition 2.5. 


Lemma 2.1 Suppose tensor F G C’^‘‘ is partial symmetric with respect to indices {l,...,m}, 
m < d. Then the tensor 

m 

i=i 

is partial symmetric with respect to indices {1,... ,m -|- 1}. Moreover, it is easy to verify that for 
i < k < m, 


'^{F- 






-F 


-k{F - + X] 


(13) 


We are now ready to prove the following key lemma. 
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Lemma 2.2 Suppose J- £ and 

r 

T = ®B'\ where i3* G C” is partial symmetric with respect to {1,, m}, m < d. 


i=l 


Then there exist tensors £ C” , which is partial symmetric with respect to {1,... ,m + 1}, for 
i = 1,...,r, such that 

r 


2=1 


Proof. Define + X] m+i 1 * From Lemma 2.1 we know that A^ is partial symmet- 


j=i 


ric with respect to {1,..., m + 1}, for i = 1,..., r. It is easy to show that 


m . 

= A^ + yq, with q = —— (b^ -b^. 


J = 1 


Because F is super-symmetric, we have F = = (.7^7ri.^+i)7rd+i,d+„,+i, which 

implies 


7- = ^ S‘ ® B- = ^ s- ® ® B‘ = ^ ® B; 


'^l,m+l * 


(14) 


2=1 


2=1 


2 = 1 


2=1 


By using (13), we have 


Bl 


A + yc. 


i=i 


■7ri,m+l 


+ - m-C). 

i=2 


(15) 


Combining (15) and (14) yields 


y B^ ® B^ 

= Eq- 

2=1 

i=i \ 

r 

r 

Ee’®si..„. = E 

2=1 

2=1 

r 

r 

yBi 

®B- = j; 

2=1 

2=1 

r 

ysi 

2=1 

® = 
•' l,m+l 


i=i 


i=i 


i=i 


i=2 


.4* + ^ q - m • C) 
i=2 


i=i 


(16) 

(17) 

(18) 


2=1 


1=2 


A + yq-m-q I. (19) 

1=2 
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It is easy to check that 


(19) + m X (18) + m X (17) + m? x (16) 

(1 + m )2 




Z=1 


i=2 


i=2 


Then we repeat this procedure. That is, since T” G , we have T" = T) 


'n'd+2,d+m+l 7r2,m + l 


= J^n 




+ 1 J'’^d+2,d+m+l 


. By letting ^ C), we can apply the same procedure as above to obtain 


i=2 


T = X]i=i S ® 1^“^* + ^jj ■ We just repeat this procedure until T" = Yl\=i 

and this completes the proof. □ 


Now we are ready to present the equivalence of symmetric M-rank and strongly symmetric M-rank. 

Theorem 2.3 For an even-order super-symmetric tensor F G its M-rank, symmetric M-rank 
and strongly symmetric M-rank are the same, i.e. rankM(.T) = rank 5 M(- 7 ^) = i’ank55M(-^)- 

Proof. The equality rankM(.T) = rank 5 M(-^) follows directly from the definition of symmetric 
M-rank. We thus only need to prove rank5M(-^) = i'ank55M(-^)- Suppose rank5M(-^) = r, which 

d . 

means there exist G C"" ,i = 1 ,... ,r, such that F = By applying Lemma 2.2 at 

i=l 

d ^ . 

most d times, we can find super-symmetric tensors G , z = 1,. .., r such that F = Yh 

i=l 

Hence, we have rank55M(-^) < r = rank^M(-^)- On the other hand, it is obvious that rank5M(-^) < 
rank^sM (-^)- Combining these two inequalities yields rank5Af(-^) = rank 55 M(- 7 ^)- Cl 


3 Bounding CP-rank for even-order tensor using M-rank 

In this section, we analyze the relation between the CP-rank and the M-rank. Specifically, for even- 
order tensor, we establish the equivalence between the symmetric CP-rank and the M-rank under 
the rank-one assumption. Then we particularly focus on the fourth-order tensors, because many 
multi-dimensional data from real practice are in fact fourth-order tensors. For example, the colored 
video completion and decomposition problems considered in [12, 14, 31] can be formulated as low- 
CP-rank fourth-order tensor recovery problems. We show that the CP-rank and the symmetric 
CP-rank for fourth-order tensor can be both lower and upper bounded (up to a constant factor) 
by the corresponding M-rank. 
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3.1 Rank-one eqnivalence for super-symmetric even-order tensor 


In our previous work [21], we already showed that if a super-symmetric even-order tensor T is 
real-valued and the decomposition is performed in the real domain, then rankcp(J^) = 1 <;=> 
rank(iW(J^)) = 1. Here we show that a similar result can be established when T is complex-valued 
and the decomposition is performed in the complex domain. To see this, we first present the 
following lemma. 

Lemma 3.1 Ij a d-th order tensor (8> ® ® G is super-symmetric, then we have 

a* = ±a^ for i = 2,... ,d and A = b<Sib0---<Sib for some b £ C^. 


Proof. Since A is super-symmetric, construct T = A® A and it is easy to show that 


'Til:.idid+i—i2d ~ ■ ■ -jd) G n(ii . . .id)i {jd+1 ■ ■ • J2d) G n(irf_|_i . . .i2d): 


and 




iil...id^d+i--i2d ~ 'Tid+\...i2dh-.idi ''^1 ^ ^ ^ ^ ^5 1 ^ id+1 < ' ' ' < * 2 d < '' T - 

Therefore, T belongs to the so-called conjugate partial symmetric tensor introduced in [20]. More¬ 
over, from Theorem 6.5 in [20], we know that 


max T(x,..., X, X,..., x) = max T(x^,..., x'^, x^,..., x'^) = 


, d ||2 


||a:||=l 


||x^|| = l, 2=l,...,d 


So there must exist an x with jjxjj = 1 such that j(a*)''~xj = for all i, which implies that 
a* = ±a^ for i = 2,... ,d, and A = A (8) <8) • • • ® for some A = ±1. Finally by taking 


b = -^a^, the conclusion follows. 


□ 


The rank-one equivalence is established in the following theorem. 

Theorem 3.2 Suppose T G and we have 

rankM(.T) = 1 <;=> rank5cp(.^) = 1- 


Proof. Suppose rank 5 c'p(J^) = 1 and T = x < 


X for some x G C"'. By constructing A = 


2d 

X 0 • • • 0 X , we have T = A® A with A G S”'*. Thus, rankM(.F') = 1. 
d 

To prove the other direction, suppose that we have T G and its M-rank is one, i.e. F = A® A 
for some A G By similar arguments as in Lemma 2.1 and Proposition 2.3 in [21], one has 
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that the Tucker rank of ^ is (1,1, • • • , 1) and consequently the asymmetric CP-rank of A is one. 
This fact together with Lemma 3.1 implies that the symmetric CP-rank of A is one as well, i.e., 
^ = 6 (g)... (g) 5 for some b G C"". It follows from T = A® A = b ® ■■■ ® b that rank 5 C'p(-^) = 1. □ 

V V 

d 2d 

3.2 Bound for asymmetric fourth-order tensor 

For an asymmetric fourth-order tensor, the relation between its CP-rank and the corresponding 
M-rank is summarized in the following result. 

Theorem 3.3 Suppose J- G ![^nixn2xn3xn4, yjHh m < n 2 < < n^. Then for any permutation n 

o/ (1, 2, 3,4) it holds that 

rank(iW(T' 7 r)) < rankcp(Jv) < nina • ra,nk{M. ( 20 ) 

Moreover, the inequalities above can be sharpened to 

lankj^+^F) < rankcp(T') < nina • rankj^^-(T"). ( 21 ) 

Proof. Suppose rankc'p(T') = r. Let the rank-one decomposition be 

r 

F = Y^ ® ® ® with G C”* for fc = 1,..., 4 and i = 1,..., r. 

i=l 

By letting ® a^’* and = a^’* ® we get F = Yll=i A'’ ® . Thus rankM(T') < r = 

rankc>p(T'). In fact, this holds for JFr where vr is any permutation of (1, 2, 3,4). 

On the other hand, for any Jv denote = rank(iW(T)r)) and {ji, j2, jsj ji) = "^(1)2,3,4). Then 
rm 

Ft, = '^A'- ® B'' with matrices A" G ^ s'- G for i = 1,..., , 

i=l 

and it follows that rank(^*) < ii and rank(B®) < £2 for alH = 1 ,..., tm, where £i := min{rej^, nj^} 
and £2 ■= minjnjg,nj 4 }. In other words, matrices vl* and admit some rank-one decompositions 
with at most £i and £2 terms, respectively. Consequently, F can be decomposed as the sum of at 
most rM£i £2 rank-one tensors, or equivalently 

rankc'p(T'7r) < oiinjnj^,• minjnjg, 71 ^ 4 } • rankM(T)r) < rains • rankM(T'7r). 

Since the bounds (20) hold for all and 

rankjv.f-(T") = minrank(iW(T'7r)), rank;y^+(T') = maxrank(iW(T'7r)), 

TT TT 
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the sharper bounds (21) follow immediately. 


□ 


The following results further show that the bounds in (21) are actually tight. 

Proposition 3.4 Let us consider a fourth order tensor T G ![^nixn2xn3xn4 fJiQf 

F = A® B for some matrices A G and B G (22) 

Denote ri = rank(^), r 2 = rank(i?). Then, the following holds: 

(i) The Tucker rank of F is (ri, ri, r 2 , r 2 ); 

(a) ia,'Qkj 4 ^+{F) = ri r 2 and rank^-(T') = 1; 

(Hi) rankcp(T') = rir 2 . 

Proof. Suppose the singular value decompositions of A and B are given by 

ri r2 

A = '^2 ® B and B = cf ® d). (23) 

i=l j=l 

Recall that F(l) denotes the mode-1 unfolding of F. According to (23), it is easy to verify that 

ri 

^■(1) = E o)®V{P®B). (24) 

i=l 

Moreover we observe that for i ^ j, {V{P ® B))^ {V (Id ® B)) = ■ tr {B~^B) = 0. Thus, (24) 

is indeed an orthogonal decomposition of F{1) and thus, rank(T(l)) = ri. Similarly we can show 
that rank(T(2)) = ri, rank(T(3)) = r 2 and rank(T(4)) = r 2 . This proves part (i). 

Now we consider the square unfoldings of F. Let F(l,2), T(l,3), T(l,4) be the square unfolded 
matrices by grouping indices (1,2), (1,3), (1,4) respectively. Due to (22), we immediately have 
that rank(T(l,2)) = 1 and also 

ri r2 

F(l,3) = EE V{a^ ®d)®V{F ®F). (25) 

i=l j=l 

From the orthogonality of a*’s, 6®’s, c^ ’s, and d-^ ’s, it follows that {V{od ® c))}ij and {V{F ® d^)}ij 
are two orthogonal bases. In other words, (25) is an orthogonal decomposition of T(l,3) and 
thus rank(T(l, 3)) = ri r 2 . In the same vein we can show that rank(T’(l, 4)) = ri r 2 as well. Since 
F(l, 2), F{1, 3) and T(l, 4) form all the square unfoldings of F, we can conclude that rankjy^+ {F) = 
rank(F(l,3)) = rank(T(l,4)) = ri r 2 and rank^v^f-(T") = 1. This proves part (ii). 
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Finally, since (22) also implies 


ri r2 

T = ^ ^ a* (g) 6* (g) ® dP , 

i=l j=l 

it then follows that rankcp(J^) < rir 2 . On the other hand, note that the first inequality in (20) 
holds for any square unfoldings. Combining this with rank(F’(l, 3)) = rir 2 , one concludes that 
ri r 2 is a lower bound of rankcpj^ as well, hence rankcp(J^) = ri r 2 . □ 

Remark 3.5 Now we remark that the bounds in (21) are tight. Suppose tensor T is given in the 
form of (22) with ni < 77-2 < ns < n^, and rank(A) = ni, rank(i?) = ns. According to the above 
results, we have rank^-(J^) = 1 and vscokM+i.^) = rankcp(J^) = nins, which imply that the both 
lower bound and upper bound in (21) are essentially tight. Moreover, in this example the Tucker rank 
is exactly (ni, ni, ns, ns), which in turn shows that rankjy/+ is a superior approximation o/rankcp 
at least for this example. In the numerical part, we shall present more examples for which rankcp is 
strictly larger than any component of Tucker rank but is essentially identical to rankj;^+. In addition, 
it is easy to show that by similar argument the bounds in (20) also hold for the ranks defined for 
real-valued decompositions, i.e., rank®(iVf (J>)) < i'ank®p(Jl,r) < nins • rank®(iVf (J>))- Moreover, 
for matrix M (J>) we have rank(iW(J>)) = rank'*(Ai'(J>)), thus establishing the following bounds: 

rankcp(Jv) < i'ank®p(J>) < nins • rank'^(7W(J>)) = nins • rank(iW(J>)) < nins • rankc>p(J>). 
Proposition 3.4 can be further extended to exactly compute the CP-rank for a class of tensors. 

Corollary 3.6 Consider an even order tensor iF G (^nixn 2 x — xn 2 d 

F = ® ® ® for some matrices A^ G <[yT- 2 i-i>:n 2 i_ 

Denoting ri = rank(A*) for i = 1,... ,d, we have that rankcp(J^) = rankjv/+(J^) = rir 2 • • • r^. 

3.3 Bound for super-symmetric fourth-order tensor 

Theorem 3.2 essentially states that the M-rank and the symmetric CP-rank are the same in the 
rank-one case. This equivalence, however, does not hold in general. In this subsection, we show 
that although they are not equivalent, the symmetric CP-rank of T" G can be both lower and 
upper bounded (up to a constant factor) by the corresponding M-rank. 

Before providing lower and upper bounds for the symmetric CP-rank, we need the following tech¬ 
nical results. 


15 


^ . 4 . . . 

Lemma 3.7 Suppose ^ vl* (g) vl* = G S"" with A'' = Yh ® £ C"" /or i = 1,..., r, 

*=i ii=i 

ji = 1,... ,mi, rrii < n. Then it holds that 

= YE a-^' (g) a-^' (g) a-^ * (g) a-^ * + 

r 

Z] X] 5 (a-^' (g) a-^' (g) (g) a^‘ + a-?* (g) (g) (g) + a-^' (g) (g) (g) a-^*) . 

'^“1 ji^^i 


Proof. Since T is super-symmetric, we have Tijke = = ^Ukj- Consequently, 

r 

T = 


2 = 1 

V (mi 

EE 

^=1 \h=i 

V (mi 

EE 

^=1 \h=i 

V (mi 

EE 

^=1 \h=i 

V (mi 

EE 

i=\ \u=\ 



E 

ii=i 


a-^* (g) a-^* ® a-^* (g) a?^ 


a?^ (g) a-^* ® a-^* (g) 


+ 0?^ (g) a-^* ® (g) 


ji^^i 


+ E 


(g) ® a-^* (g) a^* 


a-^* (g) ® a-^* 0 0-^*+ N a-^* (g) ® a-^* 


+ E 

ji^^i 


The conclusion follows by taking the average of (26), (27) and (28). 


(26) 

(27) 

(28) 

□ 


Lemma 3.8 Suppose ai, • • • , are m vectors and Ci, • • • ,^m are i.i.d. symmetric Bernoulli ran¬ 


dom variables, i.e., 




— 1, with probability 1 /2 

i = 1,... ,m. 

— 1, with probability 1/2 


Then it holds that 


vi=i 







: . 


i=i 


vi=i 


(28) 


E 

i=i 


® ad ® a^ ® a^ 


+ y^ [ad ® a* ® a^ (g) a-^ + a* (g) (g) a* ® a-^ + a* ® a^ ® a^ ® a*) . 

i¥=j 
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Proof. The expectation can be rewritten as 


1 ® 


I ® 

i=i 


I ® I 

vi=i 


vi=i 


Ee 

ij,k,£ 

y~] E a* ® a-^ 

i,j,k,£ 


( 30 ) 


Since ^i, • • • , are i.i.d. with E[^j] = 0, i = 1,..., m, it follows that 

E = 


1, a {i,j,k,i} = {u,u,v,v}, oi {u,v,u,v}, or {u,v,v,u}, for some tt, t>; 
0, otherwise . 


(31) 


Therefore, 




a* (g) ® 


i,j,k,£ 


ol (g) (g) (g) a-^ + (a* (g) a* (g) (g) a-^ + a* (g) (g) a* (g) + a* (g) a-^ (g) (g) a*) , 

i=i 


which combining with (30) yields (29). 


□ 


Now we are ready to present the main result of this subsection. 

4 

Theorem 3.9 For any given J- € E>"' , it holds that 

rankM(.T) < rank 5 C'p(J^) < (n + 4n^) rankM(.T). 


Proof. Let us first prove rankA^(J^) < rank 5 C'p(J^). Suppose rank 5 c'p(T') = r, i.e., 

r 

T = a!' ® a!" ® of ® of with a* G C” for z = 1,..., r. 

i=l 

By letting = a*(g)a*, we get F = X)i=i with G S"'^. Thus rank^^(JT) < r = rankscp(J^)- 

We now prove rank 5 C'p(J^) < (n + 4n^) rankM(.T). Suppose that rankA^(J^) = r, then from (2.3) 

^ . . 2 . . . 
it holds that T" = ^ ® A* with T* G §"■ . Assume A* = ^ a®J (g) a*^, m* < n, i = 1,... ,r. 

i=l j=l 

Let , • • • , , ^2i j • • • ) &mr be i.i.d. symmetric Bernoulli random variables. Then by combining 
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Lemmas 3.7 and 3.8, we have 


T 





2 ^ 

O 

i=l j=l 






(32) 


Assuming ..., ? 7 j^, are 4-wise independent (see, e.g., [1, 19]) and identical symmetric Bernoulli 
random variables for alH = 1,..., r, we have 


E 






= E 


rrii 


rtii 


rrii 

vi=i 


rrii 

vi=i 


, for i = 1,..., r. (33) 


According to Proposition 6.5 in [1], we know that the right hand side of (33) can be written as the 
sum of at most 4m| symmetric rank-one tensors. By combining (32) and (33) we further get 


^ r nii 

F = - ^ ^ ® (g) (g) + 

3 

i=l j=l 

( rrii \ 

which leads to 

r r 

rank5cp(-^) < + E 4m? <rn + Arn^ = (n + Ari?)iwkM{F). 

i=l i=l 



Ee 

2=1 




rrii 


ViF- 


□ 


4 The low-M-rank tensor recovery 

In this section, we consider the low-rank tensor recovery problems (2) and (4) with an emphasis 
on the fourth-order tensor case. In the context of tensor recovery, M^-rank takes a conservative 
attitude towards estimating the CP-rank while M“-rank is a more optimistic estimation. On the 
middle ground, one may choose to work with a pre-specihed vr. In our numerical experiments, for 
simplicity, for a fourth-order tensor F, we always choose to group the first two indices as the row 
index, and group the last two indices as the column index for square unfolding (we use M{F) to 
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denote the corresponding matrix). According to Theorems 3.3 and 3.9, by multiplying a constant 
factor, rank(iVT(J^)) can provide an upper bound for the CP-rank and the symmetric CP-rank of 
T (if it is also super-symmetric). We denote X = M{X),Y = M(y) and F = M{F). Without 
loss of generality, we replace the CP-rank in the objective of (2) and (4) by rank(X), and it follows 
from Theorem 3.3 that by minimizing rank(A), rankcp(A’) will be small as well. In other words, 
rather than solving (2) and (4), we solve the following two matrix problems 

min rank(X), s.t., L(X) = b, (34) 

JC 

and 

min rank(F)-|-A||Z||o, s.t., Y + Z = F, (35) 

where Z is a linear mapping such that L{X) = L[X). 

It is now very natural to consider the convex relaxations of the two matrix problems (34) and (35); 
i.e., we replace the rank function by the nuclear norm and replace the cardinality function by the 
(.1 norm. This results in the following two convex relaxations for (34) and (35): 

min ||Ai||*, s.t., L{X) = b, (36) 

and 

min ||yII*-|-A||Z||i, s.t., Y + Z = F. (37) 

Note that all the variables in (34) and (35) are complex-valued. Thus, the ii norm is defined as 
||Z||i := Ylij \/(Re {Zij)Y + (Im {Zij)Y. Although (34) and (35) are complex-valued problems, 
they can still be solved by the methods in [5, 33, 32, 2, 39] with minor modifications. We omit the 
details for succinctness. 

When the tensors are super-symmetric, we can impose the super-symmetric constraint and get the 
following formulation: 

min ||A|L 

s.t. L{Y) = b, M-^{X) 

where M~^{X) is the tensor whose square unfolding is X. Note that the above problem is equivalent 
to 

min ||A|L 

s.t. M-l(F)eS"^ (38) 

L{Y) = b, X = Y, 

which can be efficiently solved by the standard alternating direction method of multipliers (see the 
survey paper [4] for more details). 
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5 Numerical results 


In this section, we shall show the capability of our approach in estimating the CP-rank and solving 
the low-CP-rank tensor recovery problems via numerical experiments. 


5.1 Approximating the CP-rank via the M-rank 

In this subsection, we consider the problem of estimating the CP-rank for some structured tensors. 
We first construct tensors in the form of 

r 

r = ^ a* ® 6 * ® C* ® d* G cniXn2Xn3Xn4^ ( 39 ) 

^=1 

where = l,...,r are randomly generated. Obviously, the CP-rank of the resulting 

tensor T is less than or equal to r. For each set of (rei, 77 - 2 , ^, 3 , 774 ) and r, we randomly generate 20 
tensors according to (39). Then we compute the Tucker rank, the M'*'-rank and the M“-rank of 
each tensor, and report their average value over the 20 instances in Table I. 


CP-rank 

Tucker rank 

M+-rank 

M -rank 

Dimension 10 x 10 x 10 x 10 

r = 12, CP-rank < 12 

(10,10,10,10) 

12 

12 

Dimension 10 x 10 x 15 x 15 

r = 12, CP-rank < 12 

(10,10,12,12) 

12 

12 

Dimension 15 x 15 x 15 x 15 

r = 18, CP-rank < 18 

(15,15,15,15) 

18 

18 

Dimension 15 x 15 x 20 x 20 

r = 18, CP-rank < 18 

(15,15,18,18) 

18 

18 

Dimension 20 x 20 x 20 x 20 

r = 30, CP-rank < 30 

(20,20,20,20) 

30 

30 

Dimension 20 x 20 x 25 x 25 

r = 30, CP-rank < 30 

(20,20,25,25) 

30 

30 

Dimension 25 x 25 x 30 x 30 

r = 40, CP-rank < 40 

(25,25,30,30) 

40 

40 

Dimension 25 x 25 x 30 x 30 

r = 40, CP-rank < 40 

(30,30,30,30) 

40 

40 


Table 1: CP-rank approximation: M-rank vs Tucker rank for tensors in (39) 


From Table I, we can see that for all instances rank^+ (T) = rank^- (T) = r. Thus by Theorem 3.3, 
we can conclude that the CP-rank of these tensors is exactly r and the M-rank equals the CP-rank 
for these random instances. Moreover, since r is chosen to be larger than one dimension of the 
tensor, some components of the Tucker rank are strictly less than r. 
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In another setting, we generate tensors in the following manner 


r = (40) 

i=l 

where matrices A^, = 1 , ..., r are randomly generated in such a way that rank(A*) = rank(i?*) = 

k,i = Consequently, r/c^ is an upper bound for the CP-rank of T. From Propo¬ 

sition 3.4, we know that rankjy^+(7~) = rankcp(7~) = when r = 1 . One may wonder if 
rank^+(T) = rankcp(T) when r > 1. To this end, we let r = A: = 2,3,4, 5 and generate 20 
random instances for different choice of r, k and tensor dimensions. For each instance, we compute 
its Tucker rank, the M'’'-rank and the M“-rank, and report the average value over these 20 instances 
in Table 2. 


CP-rank 

Tucker rank 

M+-rank 

M -rank 

Dimension 10 x 10 x 10 x 10 

(*) r = fc = 2, CP-rank < 8 

(4,4,4,4) 

8 

2 

r = fc = 3, CP-rank < 27 

(9,9,9,9) 

27 

3 

r = fc = 4, CP-rank < 64 

(10,10,10,10) 

64 

4 

Dimension 10 x 10 x 15 x 15 

(*) r = fc = 2, CP-rank < 8 

(4,4,4,4) 

8 

2 

r = fc = 3, CP-rank < 27 

(9,9,9,9) 

27 

3 

r = fc = 4, CP-rank < 64 

(10,10,15,15) 

64 

4 

Dimension 15 x 15 x 20 x 20 

{*)r = k = 2, CP-rank < 8 

(4,4,4,4) 

8 

2 

r = k = 3, CP-rank < 27 

(9,9,9,9) 

27 

3 

r = fc = 4, CP-rank < 64 

(16,16,16,16) 

64 

4 

Dimension 20 x 20 x 20 x 20 

r = fc = 3, CP-rank < 27 

(9,9,9,9) 

27 

3 

r = k = 4, CP-rank < 64 

(15,15,16,16) 

64 

4 

r = k = 5, CP-rank < 125 

(20,20,20,20) 

125 

5 

Dimension 20 x 20 x 30 x 30 

r = k = 3, CP-rank < 27 

(9,9,9,9) 

27 

3 

r = fc = 4, CP-rank < 64 

(16,16,16,16) 

64 

4 

r = fc = 5, CP-rank < 125 

(20,20,25,25) 

125 

5 


Table 2; CP-rank approximation: M-rank vs Tucker rank for tensors in (40) 

From Table 2 we can see that rankjv^+(T) = rk'^ for all instances. This further implies that the 
CP-rank of the generated tensors is always equal to rk'^ and the M^-rank equals the CP-rank. 
Moreover, as shown in the rows that are marked by (*) in Table 2, the CP-rank is strictly less 
than any dimension of the tensor, but the Tucker rank is still strictly less than the CP-rank. This 
again confirms that the M'*“-rank is a much better approximation of the CP-rank compared with 
the averaged Tucker rank. 
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To summarize, results in both Table 1 and Table 2 suggest that rank;^^+(T) = rankcp(T) when 
T is randomly generated from either (39) or (40), while there is a substantial gap between the 
averaged Tucker rank and the CP-rank. Therefore, rank;v^+ is a better estimation than the Tucker 
rank for estimating the CP-rank, at least under the settings considered in our experiments. 


5.2 Synthetic data for low rank tensor optimization problems 
5.2.1 Low-rank tensor completion problems 


In this subsection we use the FPCA algorithm proposed in [33] to solve (36) for complex-valued 
fourth-order tensor completion and make comparison to the low-n-rank completion approach. The 
testing examples are generated as follows. We generate random complex-valued tensor Xq based 
on (39) with various tensor dimensions and different value of r so that the CP-rank of the resulting 
tensor is less than or equal to r. Under each setting, we generate 20 instances and randomly select 
70%, 50%, 30% of the entries as the observed ones for tensor completion. We also use the code 
in [14] to solve the low-n-rank tensor completion problem (6). We report the average of the Tucker 
rank, the average of the M'*“-rank and the average of the M“-rank of the recovered tensor X* over 
the 20 instances in Table 3. We also report the average of the relative errors for both approaches 
in Table 3, where the relative error is defined as 


RelErr := 


IIToIIf 


The CPU times reported are in seconds. From Table 3 we can see that when the CP-rank is not very 
small, the low-n-rank completion approach fails to recover the original tensor while our low-M-rank 
method works well and its relative error is usually at the order of 10“^ or 10“®. This result is not 
surprising. To illustrate this, let us take a 10 x 10 x 10 x 10 tensor as an example. In this case 
the mode-n unfolding and the square unfolding will result in a 10 x 1000 and a 100 x 100 matrices 
respectively. When the CP-rank of the underlying tensor is equal to 6, it is a relatively high rank 
for a 10 X 1000 matrix, while it is a relatively low rank for a 100 x 100 matrix. This is exactly what 
happens in the third row-block of Table 3 when the dimension of the tensor is 10 x 10 x 10 x 10 and 
the CP-rank is 6. In addition, we note that the Tucker rank is often larger than the CP-rank when 
it fails to complete the original tensor. However, the M'*'-rank and M“-rank are almost always 
equal to the CP-rank except for only two cases. This again suggests that the M-rank is a good 
approximation to the CP-rank. Moreover, the results in Table 3 also show that the low-M-rank 
tensor completion model has a much better recoverability than the low-n-rank model in terms of 
the relative error of the recovered tensors. Furthermore, we conduct similar tests for our low-M- 
rank model (36) when the CP-rank is larger than the length of one dimension of the tensor, and 
only 30% of entries are observed. The averaged results over 20 randomly generated instances for 
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different tensor dimensions are summarized in Table 4, which shows that our low-M-rank tensor 
completion model can still recovery the tensor very well. 

In another set of tests, we aim to observe the relationship between the M-rank and the symmetric 
CP-rank via solving the super-symmetric tensor completion problem (38). To this end, we randomly 
generate 20 complex-valued tensors Tq in the form (39) for different choice of tensor dimensions 
and values of r, so that the symmetric CP-rank of the resulting tensor is less than or equal to r. For 
each generated tensor, we randomly remove 60% of the entries and then solve the tensor completion 
problem (38). The results are reported in Table 5, which suggests that the original tensor is nicely 
recovered (usually with the relative error at the order of 10“®). Moreover, the M-rank and the 
symmetric CP-rank shown in the table are always identical, which implies that the M-rank remains 
a good approximation to the symmetric CP-rank for super-symmetric tensor. We also note that 
solving problem (38) is much more time consuming than solving (36), due to the super-symmetric 
constraint which is essentially equivalent to O(n^) linear constraints and is costly to deal with. 


5.2.2 Robust tensor recovery problem 


In this subsection we report the numerical results for the robust tensor recovery problem (4). We 
choose A = l/y^ni %2 in (37), and apply the alternating linearization method in [2] to solve (37). 
The low-rank tensor To is randomly generated according to formula (39), and a random complex¬ 
valued sparse tensor Zq is generated with cardinality of 0.05 • nin 2 n^n 4 . Then we set F = + Zq 

as the observed data in (4). We conduct our tests under various settings of tensor dimensions and 
CP-ranks of To- For each setting, 20 instances are randomly generated for test. We also apply the 
code in [14] to solve the low-n-rank robust tensor recovery problem: 

1 

min V||P(j)ll* + A||^||i, s.t. y + Z = F. (41) 

y,.2eC"i^”2-"X>id d 


Suppose T* and Z* are the low-rank tensor and sparse tensor returned by the algorithms. We 
define the relative error of the low-rank tensor as 

o iir-JoiiF 

while we also consider the relative error of the constraint violation 

\\y* yz* -f\\f 


RelErrAii ■= 


ll-F|h 


We report the average of these two relative errors, the Tucker rank and the M-rank of the recovered 
low-rank tensor T* over 20 instances in Table 6. Results in Table 6 suggest that in many cases, the 
low-n-rank robust tensor recovery model fails to extract the low-rank part while our low-M-rank 
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robust tensor recovery model can always recover the tensor with relative error at the order of 10“® 
or 10“^. Moreover, we observe that the M-rank of the recovered tensor 3^* is always equal to 
the CP-rank of the original low-rank tensor which is consistent with the observations in the 
low-rank tensor completion problems. 

5.3 Colored video completion and decomposition 

In this subsection, we apply the matrix completion and matrix robust PCA models (34) and (35) for 
the purpose of colored video completion and decomposition, which can be formulated as the fourth- 
order tensor recovery problems (2) and (4) respectively. A colored video hie consists of n 4 frames, 
and each frame is an image stored in the RGB format as a ni x n 2 x 3 array. As a result, hlling 
in the missing entries of the colored video and decomposing the video to static background and 
moving foreground can be regarded as low-rank tensor completion (2) and robust tensor recovery 
(4), respectively. 

In our experiment for tensor completion, we chose 50 frames from a video taken in a lobby, which 
was introduced by Li et al. in [28] . Each frame in this video is a colored image with size 128 x 160 x 3. 
The images in the hrst row of Figure 1 are three frames of the video. Basically we chose the 50 
frames such that they only contain static background, and thus the 128 x 160 x 3 x 50 fourth-order 
tensor formed by them are expected to have low rank, because the background is almost the same 
in each frame. We then randomly remove 80% of the entries from the video, and the images in the 
second row of Figure 1 are the frames after the removal. We then apply the FPCA proposed in [33] 
to solve (36) with the square unfolding matrix having the size 20480 x 150, to complete the missing 
entries in the target tensor. The images in the third row of Figure 1 are the frames recovered by 
FPCA for solving (36). We can see that we are able to recover the video very well even though 
80% entries are missing. 

In our experiment for robust tensor recovery, we chose another 50 frames from the same video in [28] . 
These frames were chosen such that the frames contain some moving foregrounds. The task in robust 
tensor recovery is to decompose the given tensor into two parts: a low-rank tensor corresponding 
to the static background, and a sparse tensor corresponding to the moving foreground. Note that 
the tensor corresponding to the moving foreground is sparse because the foreground usually only 
occupies a small portion of the frame. Thus this decomposition can be found by solving the robust 
tensor recovery problem (4). Here we again apply the alternating liearization method proposed in 
[2] to solve (35) for the task of robust tensor recovery, where A in (37) is chosen as 1/y/nin 2 and 
ni, 712 are the hrst two dimensions of the fourth-order tensor. The decomposition results are shown 
in Figure 2. The images in the hrst row of Figure 2 are frames of the original video. The images 
in the second and third rows of Figure 2 are the corresponding static background and moving 


24 




Figure 1: Results for video completion. The first row: frames of the original video; the second row: 
frames with 80% missing entries; the third row: recovered images using tensor completion. 


foreground, respectively. We can see that our approach very effectively decomposes the video, 
which is a fourth-order tensor. 


6 Concluding remarks 

In this paper, we propose some new notions of tensor decomposition for even-order tensors, which 
yield some rank definitions for tensors, namely, the M-rank, the symmetric M-rank and the strongly 
symmetric M-rank. We show that these three definitions are equivalent if the tensor under con¬ 
sideration is even-order and super-symmetric. We then show that the CP-rank and symmetric 
CP-rank of a given fourth-order tensor can be both lower and upper bounded (up to a constant 
factor) by the corresponding M-rank, which provides a theoretical foundation for using the M-rank 
as a proximity of the CP-rank in the low-CP-rank tensor recovery problems. This is encouraging 
since the M-rank is much easier to compute than the CP-rank. We then solve the low-M-rank 
tensor recovery problems using some existing methods for matrix completion and matrix robust 
PCA. The results show that our method can recover the tensors very well, confirming that the 
M-rank is a good approximation of the CP-rank in such applications. Compared to the classical 
mode-n rank, the new M-rank differs in the principle to unfold the tensor into matrices: it takes 
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Figure 2: Robust Video Recovery. The first row are the 3 frames of the original video sequence. 
The second row are the recovered background, and the last row are the recovered foreground. 


a balanced approach. Namely, for a 2m-order tensor, the M-rank unfolding groups the indices by 
mxm, while the Tucker rank folding groups the indices in the fashion of 1 x (2m — 1). It is certainly 
possibly to attempt all groupings such as k x (2m — k) with k = 1,2, ...,m, though the compu¬ 
tational costs increase exponentially with the order of the tensor. A balanced folding can also be 
extended to odd-order tensors; for an (2m-|- l)-order tensor, this may mean grouping the indices by 
m X (m + 1). For the tensors of order 3, this reduces to the traditional mode-n matricization. Also, 
the symmetry is lost when dealing with odd-order tensors. Other than that, most results in this 
paper, including CP-rank approximation and tensor recovery, can be generalized straightforwardly 
to odd-order tensors as well. 
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sampling ratio % 

low-n-rank completion 

low-M-rank completion 


Rel Err 

Tucker rank 

Rel Err 

M+-rank 

M -rank 

Dimension 10 x 10 x 10 x 10, r 

= 2, CP-rank < 2 


70% 

1.23e-005 

(2, 2, 2, 2) 

1.83e-005 

2 

2 

50% 

1.60e-005 

(2, 2, 2, 2) 

1.13e-005 

2 

2 

30% 

7.79e-002 

(10, 10, 10, 10) 

1.18e-004 

2.67 

2 

Dimension 10 x 10 x 10 x 10, r 

= 4, CP-rank < 4 


70% 

1.80e-005 

(4, 4, 4, 4) 

3.24e-005 

4 

4 

50% 

2.04e-001 

(10, 10, 10, 10) 

1.78e-005 

4 

4 

30% 

6.31e-001 

(10, 10, 10, 10) 

2.69e-004 

5.33 

4 

Dimension 10 x 10 x 10 x 10, r 

= 6, CP-rank < 6 


70% 

1.69e-001 

(10, 10, 10, 10) 

4.69e-005 

6 

6 

50% 

4.50e-001 

(10, 10, 10, 10) 

2.62e-005 

6 

6 

30% 

7.18e-001 

(10, 10, 10, 10) 

4.97e-006 

6 

6 

Dimension 15 x 15 x 15 x 15, r 

= 3, CP-rank < 3 


70% 

1.24e-005 

(3, 3, 3, 3) 

6.67e-006 

3 

3 

50% 

1.63e-005 

(3, 3, 3, 3) 

1.83e-006 

3 

3 

30% 

1.57e-003 

(3, 3, 3, 3) 

5.49e-005 

3 

3 

Dimension 15 x 15 x 15 x 15, r 

= 6, CP-rank < 6 


70% 

1.57e-005 

(6, 6, 6, 6) 

1.22e-005 

6 

6 

50% 

1.73e-001 

(15, 15, 15, 15) 

3.90e-006 

6 

6 

30% 

6.10e-001 

(15, 15, 15, 15) 

7.78e-005 

6 

6 

Dimension 15 x 15 x 15 x 15, r 

= 9, CP-rank < 9 


70% 

1.56e-001 

(15, 15, 15, 15) 

2.13e-005 

9 

9 

50% 

4.47e-001 

(15, 15, 15, 15) 

6.80e-006 

9 

9 

30% 

7.16e-001 

(15, 15, 15, 15) 

1.40e-004 

9 

9 

Dimension 20 x 20 x 20 x 20, r 

= 4, CP-rank < 4 


70% 

7.26e-006 

(4, 4, 4, 4) 

3.49e-006 

4 

4 

50% 

1.45e-005 

(4, 4, 4, 4) 

6.76e-007 

4 

4 

30% 

1.96e-005 

(4, 4, 4, 4) 

2.37e-005 

4 

4 

Dimension 20 x 20 x 20 x 20, r 

= 8, CP-rank < 8 


70% 

1.67e-005 

(8, 8, 8, 8) 

6.60e-006 

8 

8 

50% 

1.34e-001 

(20, 20, 20, 20) 

1.59e-006 

8 

8 

30% 

5.87e-001 

(20, 20, 20, 20) 

4.24e-005 

8 

8 

Dimension 20 x 20 x 20 x 20, r = 

= 12, CP-rank < 12 


70% 

1.45e-001 

(20, 20, 20, 20) 

8.58e-006 

12 

12 

50% 

4.20e-001 

(20, 20, 20, 20) 

3.46e-006 

12 

12 

30% 

6.99e-001 

(20, 20, 20, 20) 

6.75e-005 

12 

12 


Table 3; Low-M-rank tensor completion vs. low-n-rank tensor completion 
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rankcplTo) 

RelErr 

CPU 

rankjv^+lA"*) 

rank(A"*) 

Dimension 10 x 10 x 10 x 10 

r = 12, rankcp{Xo) < 12 

1.47e-005 

13.9 

12 

12 

Dimension 10 x 10 x 12 x 12 

r = 15, rankcp(To) < 15 

2.12e-005 

22.29 

15 

15 

Dimension 20 x 20 x 20 x 20 

r = 24, rankcpITo) < 24 

2.96e-006 

267.91 

24 

24 

Dimension 20 x 20 x 25 x 25 

r = 28, rankop(To) < 28 

1.73e-004 

11.26 

28 

28 

Dimension 30 x 30 x 30 x 30 

r = 35, rankcp(Ab) < 35 

6.63e-005 

56.27 

35 

35 

Dimension 35 x 35 x 40 x 40 

r = 45, rankcp(To) < 45 

4.05e-005 

138.77 

45 

45 


Table 4: Numerical results for low-CP-rank tensor completion by solving (36) 


rankscp(To) 

RelErr 

CPU 

rankM(A:’*) 

Dimension 10 x 10 x 10 x 10 

r = 8, rankscplTo) < 8 

8.83e-006 

5.96 

8 

r = 12, rankscp(A’o) < 12 

6.81e-006 

8.33 

12 

Dimension 15 x 15 x 15 x 15 

r = 8, rankscp(A'o) < 8 

8.95e-006 

64.66 

8 

r = 20, rankscp(A'o) < 20 

6.19e-006 

89.40 

20 

Dimension 20 x 20 x 20 x 20 

r = 15, rankscp(A’o) < 15 

6.63e-006 

523.54 

15 

r = 25, ranksc'p(Ab) < 25 

6.46e-006 

567.30 

25 

Dimension 25 x 25 x 25 x 25 

r = 15, rankscp(To) < 15 

4.21e-006 

1109.26 

15 

r = 30, rankscp(A’o) < 30 

3.87e-006 

2470.67 

30 


Table 5: Numerical results for low-CP-rank super-symmetric tensor completion by solving (38) 


32 







































































low-n-rank robust tensor recovery low-M-rank robust tensor recovery 


rankcp(To) 

Rel 'ErrAll 

Rel Errpp 

Tucker rank 

Rel Err All 

Rel ErrLR 

rankj,^+(To) 

rankjv^- (To) 

Dimension 10 x 10 x 10 x 10 

CP-rank < 2 

1.69e-004 

1.51e-002 

(3.85, 3.80, 3.90, 3.85) 

8.48e-007 

2.97e-007 

2 

2 

CP-rank < 4 

2.77e-004 

1.85e-002 

(5.30, 5.35, 5.15, 5.20) 

7.76e-007 

5.93e-007 

4 

4 

CP-rank < 6 

4.10e-004 

2.19e-002 

(10, 10, 10, 10) 

8.63e-007 

1.86e-006 

6 

6 

CP-rank < 8 

4.70e-004 

2.35e-002 

(10, 10, 10, 10) 

7.78e-007 

2.44e-006 

8 

8 

CP-rank < 12 

4.96e-004 

2.41e-002 

(10, 10, 10, 10) 

8.61e-007 

6.75e-006 

12 

12 


Dimension 15 x 15 x 15 x 15 


CP-rank < 3 

6.77e-005 

2.88e-003 

(5.15, 5.15, 5.25, 5.25) 

8.50e-007 

2.31e-007 

3 

3 

CP-rank < 6 

4.28e-004 

7.00e-003 

(15, 15, 15, 15) 

8.57e-007 

3.08e-007 

6 

6 

CP-rank < 9 

4.38e-004 

1.12e-002 

(15, 15, 15, 15) 

8.13e-007 

3.33e-007 

9 

9 

CP-rank < 12 

4.68e-004 

1.29e-002 

(15, 15, 15, 15) 

7.54e-007 

3.31e-007 

12 

12 

CP-rank < 18 

4.97e-004 

1.22e-002 

(15, 15, 15, 15) 

8.11e-007 

4.04e-007 

18 

18 


Dimension 20 x 20 x 20 x 20 


CP-rank < 4 

4.82e-004 

8.05e-003 

(20, 20, 20, 20) 

8.64e-007 

1.87e-007 

4 

4 

CP-rank < 8 

4.68e-004 

9.91e-003 

(20, 20, 20, 20) 

7.37e-007 

2.36e-007 

8 

8 

CP-rank < 12 

4.53e-004 

1.07e-002 

(20, 20, 20, 20) 

8.59e-007 

3.09e-007 

12 

12 

CP-rank < 16 

4.69e-004 

1.09e-002 

(20, 20, 20, 20) 

8.26e-007 

3.31e-007 

16 

16 

CP-rank < 24 

4.97e-004 

9.42e-003 

(20, 20, 20, 20) 

8.59e-007 

4.00e-007 

24 

24 


Table 6; Low-M-rank robust tensor recovery vs. low-n-rank robust tensor recovery 
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